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Purpose of Study 


• Retention is an important issue in higher education. 

• Over 50%of students leave their first 4 yr institution without receiving a 
degree and almost half of those students do not ever receive degrees (Tinto, 
1993). 

• Individuals with college degrees earned on average 62%more than those 
with only high school diplomas (Baum & Ma, 2007). 

• Students with higher SAT scores and HSGPAs are more 
likely to be retained. 

• 95.5% of students in the highest SAT score band returned for their second 
year compared to 63.8% of students in the students in the lowest SAT score 
band (Mattern & Patterson, 2009). 

• The current study explores the validity and potential of 

using the SAT, in conjunction with HSGPA, to improve 
student retention at four-year postsecondary insti ' ~ 
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What is the “Academic Index” and how is it 
related to retention? 


• Admission offices typically incorporate SAT or ACT results 
along with HSGPA and possibly other predictors of interest 
to compute an "academic index" 

• By including applicant information such as HSGPAs and SAT 
scores in the model with the appropriate weights for each 
measure, one can arrive at a predicted FYGPA which can 
signal to the institution how well the applicant should be 
able to perform at the institution. 

• This predicted FYGPA, or academic index, can be used in a 
variety of ways in admissions depending on the institution's 
philosophies and policies (Rigol, 2003). 
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Study Questions 


• By examining the difference between a student's predicted 
FYGPA and their actual FYGPA once in college, we can 
explore the role of the SAT and HSGPA as tools in identifying 
students at risk for leaving an institution. 

• We ask the following questions: 

• Are those students performing below their potential (predicted 
FYGPA) in college more likely to leave an institution (after 2, 3, or 4 
years)?; 

• Does a discrepancy between predicted and actual performance flag a 
problem? 

• What are the retention implications of the direction and/or magnitude of 
the discrepancy in the predicted versus actual FYGPA (performing much 
worse or much better than predicted)? 
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Sample 


• National, longitudinal higher education database which 
included first-time, first-year students that entered 1 10 four- 
year colleges/universities in fall 2006. 

• 59 of the original 1 1 0 institutions provided students' coursework, 
grades, and retention to the fourth-year. 

• The sample of institutions was diverse with regard to region, control, 
size, and selectivity (admittance rate). 

• Data used to calculate a student's academic index were 
available for 75,847 students 

• Logistic regression was used to model retention. The model included 
additional predictors (gender, race/ethnicity, parental income), 
resulting in a final sample size of 48,398 students. 
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Measures 


• SAT scores 

•High School GPA (HSGPA) 

• Demographic information 

• Institutional characteristics 
•First-Year GPA (FYGPA) 

• College retention 
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Analyses 


• The extent to which students over- or under-performed in 
their first-year of college was determined based on the 
difference between a student's predicted FYGPA (based 
on running a regression within each institution with 
HSGPA and SAT scores included as predictors) and 
observed FYGPA, which is referred to as residual or error 
term: 


Residual = FYGPA observed - FYGPA predicted 
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Retention Rate 


Analyses (cont.) 


Second-, Third-, and Fourth-Year Retention Rates by FYGPA Residual 



■2ndyr 


3rdyr 4th yr 


Both students who earned college grades that were higher (positive residuals) and 
lower (negative residuals) than what was expected given their academic credentials 
(i.e., SAT scores and HSGPA) were less likely to return for their second, third, and 
fourth year. 

o 
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Analyses (cont.) 


• Based on visual inspection of the relationship, logistic 
regressions predicting 2 nd , 3 rd , and 4 th year retention were 
run with the absolute value of the residual included as a 
predictor: 

|residual| — | F YG — F YG PAp re( jj C ^ e( j 

• FYGPA, gender, race/ethnicity, and socio-economic status 
and institutional characteristics such as selectivity and 
control were included as controls in the model. 

• Interactions (student and institutional characteristics x 
absolute residual) were included to examine whether under- 
or over-performing has more of an effect for certain groups 
of students or types of institutions. 
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Results - Descriptive Statistics 


Subgroup 


N 

FYGPA 

M 

observed 

SD 

FYGPA 

M 

predicted 

SD 

| residual | 
M SD 

2 nd Yr 
Retention 
M SD 

Overall 


48,398 

2.97 

.71 

2.98 

.37 

.46 

.39 

.87 

.33 

Ethnicity 

Underrepresented minority 

7,382 

2.69 

.74 

2.81 

.38 

.51 

.43 

.85 

.36 

Nonminority 

41,016 

3.02 

.69 

3.01 

.36 

.45 

.38 

.88 

.33 

Gender 

Female 

25,963 

3.05 

.67 

2.99 

.37 

.44 

.36 

.87 

.34 

Male 

22,435 

2.88 

.73 

2.96 

.38 

.48 

.41 

.88 

.33 


< $30,000 

5,266 

2.79 

.77 

2.86 

.38 

.51 

.45 

.84 

.36 


$30,000 - $50,000 

6,739 

2.89 

.73 

2.93 

.38 

.48 

.40 

.85 

.35 

Income 

$50,000 - $70,000 

7,756 

2.93 

.71 

2.96 

.37 

.47 

.39 

.86 

.35 


$70,000 -$100,000 

12,263 

2.99 

.70 

2.98 

.37 

.46 

00 

CO 

.87 

.33 


>$100,000 

16,374 

3.06 

.66 

3.04 

.36 

.42 

.36 

.90 

.30 

Institutional 

Private 

17,015 

3.11 

.59 

3.11 

.33 

.37 

.31 

.91 

.28 

Control 

Public 

31,383 

2.89 

.75 

2.91 

.37 

.50 

.42 

.85 

.36 

Institutional 

Under 50% 

7,024 

3.24 

.50 

3.25 

.25 

.33 

.27 

.95 

.22 

Selectivity 

50 to 75% 

31,515 

2.93 

.72 

2.94 

.3b 

.48 

.40 

.87 

.34 

Over 75% 

9,859 

2.89 

.75 

2.90 

.41 

.48 

.40 

.82 

.38 
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Results -Logistic Regression for Retention to 2 nd Yr 



B 

00 

m 

Exp(B) 

Sig. 

VIF 

Pseudo Pseudo 

R 2 R 2 

n CS n N 

Constant 

2.509 

.052 

12.298 

<.001 


.108 .202 

FYGPA 

.797 

.023 

2.218 

<.001 

1.338 


|Residual| 

-.868 

.084 

.420 

<.001 

6.030 


Underrepresented minority 

.235 

.071 

1.265 

.001 

2.598 

/ Str °ngest 1 
Predictors / 

Underrepresented X |Residual| 

-.188 

.088 

.829 

.033 

2.938 

fon s t / 

Male 

.426 

.050 

1.530 

<.001 

2.409 


Gender X |Residual| 

-.233 

.067 

.792 

<.001 

3.708 


Public 

-.067 

.059 

.935 

.259 

3.218 


Public X |Residual| 

.044 

.085 

1.045 

.604 

7.776 


Selectivity 

-1.845 

.213 

.158 

<.001 

3.165 


Selectivity X |Residual| 

-.503 

.311 

.605 

.106 

3.001 


Income 

.049 

.018 

1.051 

.008 

2.578 


Income X |Residual| 

-.021 

.024 

.979 

.390 

2.637 



Note. N - 48,398. B = log odds; Exp(B) = odds ratio; Pseudo RCS2= Cox and Snell R2\ Pseudo RN2= Nagelkerke R2. 
Selectivity = numbers of students admitted/ number of student that applied. FYGPA, Selectivity, and Income were grand mean 
centered. Underrepresented minority students include American Indian, African American, and Hispanic students. Nonminority 
students include White and Asian students. 

12 [ CollegeBoard 

inspiring mindd" 



Implications of Study Findings 


• The residual indicates that the student is behaving academically 
differently than expected and is likely signaling that the student 
is at risk for leaving the institution. 

• The residual likely holds a great deal of information related to psycho-social 
variables at play, which are typically very hard to obtain and/or measure in 
higher education. 

• By capturing students' predicted FYGPAs during admission and 
then later comparing them to the students' actual/observed 
FYGPAs or even first-semester GPAs, the IR or EM office can 
develop a list of students that they may choose to contact for 
follow-up academic or other counseling services. 

• Students who perform much better than predicted are also at 
greater risk for leaving an institution. 
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Limitations 


• Used self-reported HSGPA. 

• Though prior research suggests that students are quite 
accurate in reporting their HSGPA (e.g. Kuncel, Crede, & 
Thomas, 2005; Shaw & Mattern, 2009) 

• Access to "reasons" as to why students left 
institutions would inform future research. 

• Examine graduation as the outcome when available. 
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Future Research 


• Higher Education Trajectories for Over- versus under- 
performers 

• Drop out versus transfer? 

• Transfer up versus transfer down? 

• Better understand factors influencing under-performance, and 
examine whether the predicted or actual outcome was the 
truer reflection of the student's capabilities. 

• Replicate and explore interventions to assist these students in 
adjusting to their institution and/or choosing other academic 
pursuits that may better fit their needs and interests. 


15 



inspiring mindS" 


Thank You 


• Researchers are encouraged to freely express their 
professional judgment. Therefore, points of view or 
opinions stated in College Board presentations do not 
necessarily represent official College Board position or 
policy. 

• Please forward any questions, comments, and 
suggestions to: 

• Emily Shaw at eshaw@colleaeboard.org 
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